Full-color visibility model using csf which varies spatially with local luminance

ABSTRACT

The present disclosure relate generally to color science and digital watermarking. A full color visibility model has been developed which has good correlation to subjective visibility tests for color patches degraded with a watermark. A relatively better correlation can be achieved with a model that applies a luminance correction to the contrast sensitivity functions (CSF). The model can be applied during the watermark embed process, using a pyramid based method, to obtain equal visibility. Better robustness and visibility can be obtained with equal visibility embed than uniform strength embed. Of course, other features, combinations and claims are disclosed as well.

RELATED APPLICATION DATA

This patent application claims the benefit of U.S. Provisional PatentApplication No. 61/923,060, filed Jan. 2, 2014, which is herebyincorporated herein by reference.

This application is related to U.S. Pat. No. 8,199,969, US PublishedPatent Application No. US 2010-0150434 A1; U.S. patent application Ser.No. 13/975,919, filed Aug. 26, 2013; and U.S. Provisional ApplicationNos. 61/693,106, filed Aug. 24, 2012; 61/716,591, filed Oct. 21, 2012;and 61/719,920, filed Oct. 29, 2012; and 61/749,767, filed Jan. 7, 2013.

The above patent documents are each hereby incorporated herein byreference in their entirety.

TECHNICAL FIELD

The present disclosure relates generally to color science, imageprocessing, steganographic data hiding and digital watermarking.

BACKGROUND AND SUMMARY

The term “steganography” generally means data hiding. One form of datahiding is digital watermarking. Digital watermarking is a process formodifying media content to embed a machine-readable (ormachine-detectable) signal or code into the media content. For thepurposes of this application, the data may be modified such that theembedded code or signal is imperceptible or nearly imperceptible to auser, yet may be detected through an automated detection process. Mostcommonly, digital watermarking is applied to media content such asimages, audio signals, and video signals.

Digital watermarking systems may include two primary components: anembedding component that embeds a watermark in media content, and areading component that detects and reads an embedded watermark. Theembedding component (or “embedder” or “encoder”) may embed a watermarkby altering data samples representing the media content in the spatial,temporal or some other domain (e.g., Fourier, Discrete Cosine or Wavelettransform domains). The reading component (or “reader” or “decoder”)analyzes target content to detect whether a watermark is present. Inapplications where the watermark encodes information (e.g., a message orpayload), the reader may extract this information from a detectedwatermark.

A watermark embedding process may convert a message, signal or payloadinto a watermark signal. The embedding process then combines thewatermark signal with media content and possibly another signals (e.g.,an orientation pattern or synchronization signal) to create watermarkedmedia content. The process of combining the watermark signal with themedia content may be a linear or non-linear function. The watermarksignal may be applied by modulating or altering signal samples in aspatial, temporal or some other transform domain.

A watermark encoder may analyze and selectively adjust media content togive it attributes that correspond to the desired message symbol orsymbols to be encoded. There are many signal attributes that may encodea message symbol, such as a positive or negative polarity of signalsamples or a set of samples, a given parity (odd or even), a givendifference value or polarity of the difference between signal samples(e.g., a difference between selected spatial intensity values ortransform coefficients), a given distance value between watermarks, agiven phase or phase offset between different watermark components, amodulation of the phase of the host signal, a modulation of frequencycoefficients of the host signal, a given frequency pattern, a givenquantizer (e.g., in Quantization Index Modulation) etc.

The present assignee's work in steganography, data hiding and digitalwatermarking is reflected, e.g., in U.S. Pat. Nos. 6,947,571; 6,912,295;6,891,959. 6,763,123; 6,718,046; 6,614,914; 6,590,996; 6,408,082;6,122,403 and 5,862,260, and in published specifications WO 9953428 andWO 0007356 (corresponding to U.S. Pat. Nos. 6,449,377 and 6,345,104).Each of these patent documents is hereby incorporated by referenceherein in its entirety. Of course, a great many other approaches arefamiliar to those skilled in the art. The artisan is presumed to befamiliar with a full range of literature concerning steganography, datahiding and digital watermarking.

One possible combination of the inventive teaching is a methodincluding: receiving a color image or video; transforming the colorimage or video signal by separating the color image or video into atleast first data representing a first color channel of the color imageor video and second data representing a second color channel of thecolor image or video, where the first data comprises a digital watermarksignal embedded therein and the second data comprises the digitalwatermark signal embedded therein with a signal polarity that isinversely related to the polarity of the digital watermark signal in thefirst data; subtracting the second data from the first data to yieldthird data; using at least a processor or electronic processingcircuitry, analyzing the third data to detect the digital watermarksignal; once detected, providing information associated with the digitalwatermark signal.

Another combination is a method including: obtaining first datarepresenting a first chrominance channel of a color image or video,where the first data comprises a watermark signal embedded therein;obtaining second data representing a second chrominance channel of thecolor image or video, the second data comprising the watermark signalembedded therein but with a signal polarity that is inversely related tothe polarity of the watermark signal in the first data; combining thesecond data with the first data in manner that reduces image or videointerference relative to the watermark signal, said act of combiningyielding third data; using at least a processor or electronic processingcircuitry, processing the third data to obtain the watermark signal;once obtained, providing information associated with the watermarksignal.

Still another combination is an apparatus comprising: a processor orelectronic processing circuitry to control: (a) handling of first datarepresenting a first color channel of a color image or video, where thefirst data comprises a watermark signal embedded therein; (b) handlingof second data representing a second color channel of the color image orvideo, the second data comprising the watermark signal embedded thereinbut with a signal polarity that is inversely related to the polarity ofthe watermark signal in the first data; (c) combining the second datawith the first data in manner that reduces image or video interferencerelative to the watermark signal, the combining yielding third data; (d)processing the third data to obtain the watermark signal; and (e) onceobtained, providing information associated with the watermark signal.

Yet another possible combination is a method including: a methodincluding: obtaining first data representing a first chrominance channelof a color image or video signal; obtaining second data representing asecond chrominance channel of the color image or video signal; using aprocessor or electronic processing circuitry, embedding a watermarksignal in the first data with a first signal polarity; using a processoror electronic processing circuitry, transforming the second data byembedding the watermark signal in the second data so that when embeddedin the second data the watermark signal comprises a second signalpolarity that is inversely related to the first signal polarity of thewatermark signal in the first data; combining the watermarked first dataand the watermarked second data to yield a watermarked version of thecolor image or video signal, whereby during detection of the watermarksignal from the watermarked version of the color image or video signal,the second data is combined with the first data in a manner that reducesimage or video signal interference relative to the watermark signal.

Still a further combination is a digital watermarking method comprising:using a programmed electronic processor, modeling a first color ink anda second color ink in terms of CIE Lab values; modulating the valueswith a watermarking signal; scaling the modulated values in a spatialfrequency domain; spatially masking the scaled, modulated values;providing the spatially masked, scaled, modulated values, such valuescarrying the watermark signal.

Further combinations, aspects, features and advantages will become evenmore apparent with reference to the following detailed description andaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The patent or application file contains at least one drawing executed incolor. Copies of this patent or patent application publication withcolor drawing(s) will be provided by the Office upon request and paymentof the necessary fee.

FIG. 1 represents a color image.

FIG. 2 represents a first color channel (‘a’ channel) of the color imagerepresentation shown in FIG. 1.

FIG. 3 represents a second color channel (‘b’ channel) of the colorimage representation shown in FIG. 1.

FIG. 4 is a representation of the sum of the first color channel of FIG.2 and the second color channel of FIG. 3 (e.g., a+b).

FIG. 5 is a graph showing a histogram standard deviation of FIG. 4.

FIG. 6 is a representation of the difference between the first colorchannel of FIG. 2 and the second color channel of FIG. 3 (a−b).

FIG. 7 is a graph showing a histogram standard deviation of FIG. 6.

FIG. 8 is an image representation of the difference between the firstcolor channel of FIG. 2 (including a watermark signal embedded therein)and the second color channel of FIG. 3 (including the watermark signalembedded therein).

FIG. 9 is a graph showing a histogram standard deviation of FIG. 8.

FIGS. 10 a and 10 b are block diagrams showing, respectively, anembedding process and a detection process.

FIG. 11 is a diagram showing watermarks embedded in first and secondvideo frames.

FIG. 12 is a diagram showing a detailed signal size view with inkincrements of 2%, and the addition of press visibility constraints.

FIG. 13 a corresponds to Appendix D's FIG. 1, which shows a qualityruler increasing in degradation from B (slight) to F (strong).

FIG. 13 b corresponds to Appendix D's FIG. 2, which shows thumbnails ofthe 20 color patch samples with a watermark applied.

FIG. 14 corresponds to Appendix D's FIG. 3, which shows a mean observerresponses with 95% confidence intervals for color patches.

FIG. 15 corresponds to Appendix D's FIG. 4, which shows mean observerresponse compared with a proposed visibility model.

FIG. 16 corresponds to Appendix D's FIG. 5, which shows mean observerresponse compared with the proposed visibility model with luminanceadjustment.

FIG. 17 corresponds to Appendix D's FIG. 6, which shows mean observerresponse compared with S-CIELAB.

FIG. 18 corresponds to Appendix D's FIG. 7, which shows watermarkembedding with uniform signal strength (left) and equal visibility froma visibility model (right). The insets are magnified to show imagedetail.

FIG. 19 corresponds to Appendix D's FIG. 8, which shows visibility mapfrom uniform signal strength embedding (left) and equal visibilityembedding (right) from FIG. 18.

FIG. 20 corresponds with Appendix D's FIG. 9, which shows Apple tart,Giraffe stack and Pizza puff design used in tests.

DETAILED DESCRIPTION

Portions of the following disclosure discusses a digital watermarkingtechnique that utilizes at least two chrominance channels (also called“color planes,” “color channels” and/or “color direction”). Chrominanceis generally understood to include information, data or signalsrepresenting color components of an image or video. In contrast to acolor image or video, a grayscale (monochrome) image or video has achrominance value of zero.

Media content that includes a color image (or color video) isrepresented in FIG. 1. An industry standard luminance and chrominancecolor space is called “Lab” (for Lightness (or luminance), plus ‘a’ and‘b’ color channels) that can be used to separate components of imagesand video. FIG. 2 is an ‘a’ channel representation of FIG. 1 (shown ingrayscale), and FIG. 3 is a ‘b’ channel representation of FIG. 1 (shownin grayscale). Of course, our inventive methods and apparatus will applyto and work with other color schemes and techniques as well. Forexample, alternative luminance and chrominance color schemes include“Yuv” (Y=luma, and ‘u’ and ‘v’ represent chrominance channels) and“Ycc.” (also a dual chrominance space representation).

Let's first discuss the additive and subtractive effects on FIGS. 2 and3. FIG. 4 illustrates a representation of the result of adding the ‘a’channel (FIG. 2) with the ‘b’ channel (FIG. 3). FIG. 6 illustrates arepresentation of the result of subtracting the ‘b’ channel (FIG. 3)from the ‘a’ channel (FIG. 2). The result of subtracting the ‘b’ channelfrom the ‘a’ channel yields reduced image content relative to adding thetwo channels since the ‘a’ and ‘b’ color planes have correlated imagedata in the Lab scheme. (In typical natural imagery, the ‘a’ and ‘b’chrominance channels tend to be correlated. That is to say where ‘a’increases, ‘b’ also tends to increase. One measure of this is to measurethe histogram of the two chrominance planes when they are added (seeFIG. 5), and compare that to the histogram when the two color planes aresubtracted (see FIG. 7). The fact that the standard deviation of FIG. 7is about half that of FIG. 5 also supports this conclusion, andillustrates the reduction in image content when ‘b’ is subtracted from‘a’) In this regard, FIG. 4 provides enhanced or emphasized imagecontent due to the correlation. Said another way, the subtraction of theFIG. 3 image from FIG. 2 image provides less image interference orreduces image content. The histogram representations of FIG. 4 and FIG.6 (shown in FIGS. 5 and 7, respectively) further support thisconclusion.

Now let's consider watermarking in the context of FIGS. 2 and 3.

In a case where a media signal includes (or may be broken into) at leasttwo chrominance channels, a watermark embedder may insert digitalwatermarking in both the ‘a’ color direction (FIG. 2) and ‘b’ colordirection (FIG. 3). This embedding can be preformed in parallel (ifusing two or more encoders) or serial (if using one encoder). Thewatermark embedder may vary the gain (or signal strength) of thewatermark signal in the ‘a’ and ‘b’ channel to achieve improved hidingof the watermark signal. For example, the ‘a’ channel may have awatermark signal embedded with signal strength that greater or less thanthe watermark signal in the ‘b’ channel. Alternatively, the watermarksignal may be embedded with the same strength in both the ‘a’ and ‘b’channels. Regardless of the watermark embedding strength, watermarksignal polarity is preferably inverted in the ‘b’ color plane relativeto the ‘a’ color plane. The inverted signal polarity is represented by aminus (“−”) sign in equations 1 and 2.

WMa=a(channel)+wm  (1)

WMb=b(channel)−wm  (2)

WMa is a watermarked ‘a’ channel, WMb is a watermarked ‘b’ channel, andwm represents a watermark signal. A watermarked color image (including Land WMb and WMa) can be provided, e.g., for printing, digital transferor viewing.

An embedded color image is obtained (from optical scan data, memory,transmission channel, etc.), and data representing the color image iscommunicated to a watermark detector for analysis. The detector (or aprocess, processor or electronic processing circuitry used inconjunction with the detector) subtracts WMb from WMa resulting in WMresas shown below:

WMres=WMa−WMb  (3)

WMres=(a+wm)−(b−wm)  (4)

WMres=(a−b)+2*wm  (5)

This subtraction operation yields reduced image content (e.g., FIG. 6)as discussed above. The subtraction or inverting operation of the colorchannels also emphasizes or increases the watermark signal (2*wm),producing a stronger watermark signal for watermark detection. Indeed,subtracting the color channels increases the watermark signal-to-mediacontent ratio: WMres=(a−b)+2*wm.

FIG. 8 illustrates the result of equation 5 (with respect to watermarkedversions of FIG. 2 and FIG. 3). As shown, the perceptual “graininess” or“noise” in the image corresponds to the emphasized watermark signal. Theimage content is also reduced in FIG. 8. A histogram representation ofFIG. 8 is shown in FIG. 9 and illustrates a favorable reduction of imagecontent.

A watermark detector may extract or utilize characteristics associatedwith a synchronization signal (if present) from a frequency domainrepresentation of WMres. The detector may then use this synchronizationsignal to resolve scale, orientation, and origin of the watermarksignal. The detector may then detect the watermark signal and obtain anymessage or payload carried thereby.

To even further illustrate the effects of improving the watermarksignal-to-media content ratio with our inventive processes and systems,we provide some additive and subtractive examples in the content ofwatermarking.

For the following example, a watermark signal with the same polarity isembedded in each of the ‘a’ color channel and the ‘b’ color channel. Thesame signal polarity is represented by a plus (“+”) sign in equations 6and 7.

WMa=a+wm  (6)

WMb=b+wm  (7)

WMa is a watermarked ‘a’ channel, WMb is a watermarked ‘b’ channel, andwm represents a watermark signal. A watermarked color image (including Land WMb and WMa) can be provided, e.g., for printing, digital transferor viewing.

An embedded color image is obtained, and data representing the colorimage is communicated to a watermarked detector for analysis. Thedetector (or a process, processor, or electronic processing circuitryused in conjunction with the detector) adds the ‘a’ and ‘b’ colorchannels to one another (resulting in WMres) as shown below:

WMres=WMa+WMb  (8)

WMres=(a+wm)+(b+wm)  (9)

WMres=(a+b)+2*wm  (10)

This addition operation results in increased image content (e.g., FIG.4). Indeed, image interference during watermark detection will begreater since the two correlated ‘a’ and ‘b’ color channels tend toreinforce each other.

By way of further example, if WMb is subtracted from WMa (with watermarksignals having the same polarity), the following results:

WMres=WMa−WMb  (11)

WMres=(a+wm)−(b+wm)  (12)

WMres=(a−b)+≈0*wm  (13)

A subtraction or inverting operation in a case where a watermark signalincludes the same polarity decreases image content (e.g., FIG. 4), butalso significantly decreases the watermark signal. This may result inpoor—if any—watermark detection.

FIGS. 10 a and 10 b are flow diagrams illustrating some relatedprocesses and methods. These processes may be carried out, e.g., via acomputer processor, electronic processing circuitry, printer, handhelddevice such as a smart cell phone, etc.

With reference to FIG. 10 a, a color image (or video) is obtained andseparated into at least two (2) color channels or planes (10). Awatermark signal is determined for the color image or video (12). Ofcourse, the watermark signal for the color image or video may bedetermined prior to or after color plane separation. The determinedwatermark signal is embedded in a first of the color planes (14). Aninverse polarity version of the watermark signal is embedded in a secondcolor plane. The color planes are recombined (perhaps with datarepresenting luminance) to form a composite color image.

With reference to FIG. 10 b, a watermarked color image or video isobtained or received (11). The color image (or video) has or can beseparated into at least two (2) color planes or channels (13). A firstcolor plane includes a watermark signal embedded therein. A second colorplane includes the watermark signal embedded therein with a polaritythat is inversely related to the watermark signal in the first colorplane. The watermarked second color plane is subtracted from thewatermarked first color (15). The result of the subtraction is analyzedto detect the watermark signal. A detected watermark message, signal orpayload can be provided (19), e.g., to a remote database to obtainrelated metadata or information, to a local processor, for display, to arights management system, to facilitate an online transaction, etc.

In addition to the Lab color scheme discussed above, a watermark signalmay be embedded in color image (or video) data represented by RGB, Yuv,Ycc, CMYK or other color schemes, with, e.g., a watermark signalinserted in a first chrominance direction (e.g., red/green direction,similar to that discussed above for the ‘a’ channel) and a secondchrominance direction (e.g., a blue/yellow direction, similar to thatdiscussed above for the ‘b’ channel). For watermark signal detectionwith an alterative color space, e.g., an RGB or CMYK color space, animage can be converted to Lab (or other color space), or appropriateweights of, e.g., RGB or CMY channels, can be used. For example, thefollowing RGB weights may be used to calculate ‘a’−‘b’: ChrominanceDifference=0.35*R−1.05*G+0.70*B+128, where R, G and B are 8-bitintegers.

Further Considerations of Video

The human contrast sensitivity function curve shape with temporalfrequency (e.g., relative to time) has a very similar shape to thecontrast sensitivity with spatial frequency.

Successive frames in a video are typically cycled at about at least 60Hz to avoid objectionable visual flicker. So-called “flicker” is due tothe high sensitivity of the human visual system (HVS) to high temporalfrequency changes in luminance. The human eye is about ten (10) timesless sensitive to high temporal frequency chrominance changes.

Consider a video sequence with frames as shown in FIG. 11. A chrominancewatermark can be added to frame 1 per the above description for images.In a similar way, a watermark is added to frame 2 but the polarity isinverted as shown in FIG. 11.

In order to recover the watermark, pairs of frames are processed by awatermark detector, and the ‘a’ channels are subtracted from each otheras shown below.

Det _(—) a=(a1+wm)−(a2−wm)=(a1−a2)+2*wm  (14)

Det_a refers to watermark detection processing of the ‘a’ channel.Because of the temporal correlation between frames, the image content inequation 14 is reduced while the watermark signal is reinforced.

In a similar way the ‘b’ channels are also subtracted from each other

Det _(—) b=(b1−wm)−(b2+wm)=(b1−b2)−2*wm  (15)

Det_a refers to watermark detection processing of the ‘b’ channel.Equation 14 and 15 are then subtracted from each other as shown below inequation 16.

$\begin{matrix}{\begin{matrix}{{{Det\_ a} - {Det\_ b}} = {\left( {{a\; 1} - {a\; 2} + {2*{wm}}} \right) - \left( {{b\; 1} - {b\; 2} - {2*{wm}}} \right)}} \\{= {\left( {{a\; 1} - {a\; 2}} \right) - \left( {{b\; 1} - {b\; 2}} \right) + {4*{wm}}}}\end{matrix}\quad} & (16)\end{matrix}$

In generally, related (but not necessarily immediately adjacent) frameswill have spatially correlated content. Because of the spatialcorrelation between the ‘a’ and ‘b’ frames, the image content is reducedwhile the watermark signal is reinforced. See equation 16.

For any one pair of frames selected by a watermark detector, thepolarity of the watermark could be either positive or negative. To allowfor this, the watermark detector may examine both polarities.

Watermark Embedding for Spot Colors

Product packaging is usually printed in one of two ways:

1. Process color printing using cyan, magenta yellow and/or black (CMYK)

2. Spot color printing (e.g., using special Pantone color or other inksets)

The majority of packaging is printed using spot colors mainly forreasons of cost and color consistency, and to achieve a wide color gamutover various packaging. Some conventional watermarking techniques embeddigital watermarks in either CMYK for printed images or RGB for digitalimages that are being displayed. But how to embed a watermark with aspot color?

An improvement addresses problem associated with watermarking spot colorimages. Preferably, packaging contains two (2) or more spot colors(e.g., printed cooperatively to achieve a certain color consistency).Each different color is altered to collectively carry a watermarksignal. A maximum signal strength within a user selectable visibilityconstraint with watermark in at least two (2) of the spot.

A maximized watermark signal is embedded preferably by modulating thespot color inks within a certain visibility constraint across the image.The approach models a color (ink) in terms of CIE Lab values. Lab is auniform perceptual color space where a unit difference in any colordirection corresponds to an equal perceptual difference.

The Lab axes are then scaled for the spatial frequency of the watermarkbeing added to the image, in a similar manner to the Spatial CieLabmodel by X. Zhang and B. A. Wandell, e.g., “A spatial extension ofCIELAB for digital color image reproduction,” in Proceedings of theSociety of Information Display Sumposium (SID '96), vol. 27, pp.731-734, San Jose, Calif., USA, June 1996. This is a uniform perceptualcolor space which we will call SLAB, where a unit difference in anycolor direction corresponds to an equal perceptual difference due to theaddition of a watermark signal at that spatial frequency.

The allowable visibility magnitude in SLAB is scaled by spatial maskingof the cover image. Spatial masking of the cover image can include thetechniques described by Watson in US Published Patent Application No. US2006-0165311 A1, which is hereby incorporated by reference in itsentirety, and can be used to scale the allowable visibility across theimage. This is a uniform perceptual color space which we will call VLAB,where the visibility circle is scaled to correspond to an equalperceptual difference due to the addition of a watermark signal at thatspatial frequency for that particular image.

The chrominance embedding techniques discussed above forms thefoundation for the present watermark embedding techniques. Bradley etal. (Appendix A, which is hereby incorporated by reference in itsentirety from U.S. patent application Ser. No. 13/975,919, filed Aug.26, 2013) further developed this work to use an iterative embedtechnique to insert the maximum watermark signal into CMYK images.

The spot color technique described extends this work to embedding thatsupports special color inks (e.g., spot colors) used in packaging anduses a full color visibility model with spatial masking. A geometricenumerated embed approach can be used to evaluate a range of possibleink changes, which meet the user selected visibility constraint andpress constraints. The set of allowable ink changes are evaluated tochoose the pair of ink changes which result in the maximum signalstrength while meeting the visibility and press constraints.

FIG. 12 shows a detailed signal size view with ink increments of 2%, andthe addition of press constraints.

A user can insert a maximum watermark signal, while meeting anypre-required visibility constraint. The method has been applied to thecase of two spot colors and images have been produced which are morethan twice as robust to Gaussian noise as a single color image which isembedded using a luminance only watermark to the same visibility.

A method has been described which allows an image containing 2 or morespot colors to be embedded with a watermark in 2 of the spot colors,with the maximum signal strength within a user selectable visibilityconstraint.

A look-up table based approach can be used for given colors at givenlocations, and can easily be extended to 3 or more dimensions whilestill being computationally reasonable.

Additional related disclosure is found in Appendix B & Appendix C, whichare each hereby incorporated herein by reference in its entirety fromU.S. patent application Ser. No. 13/975,919, filed Aug. 26, 2013.

Full-Color Visibility Model

A full color visibility model has been developed that uses separatecontrast sensitivity functions (CSFs) for contrast variations inluminance and chrominance (red-green and blue-yellow) channels. Thewidth of the CSF in each channel can be varied spatially depending onthe luminance of the local image content. The CSF can be adjusted sothat relatively more blurring occurs as the luminance of the localregion decreases. The difference between the contrast of the blurredoriginal and marked image can be measured using a color differencemetric.

This spatially varying CSF performed better than a fixed CSF in thevisibility model, approximating subjective measurements of a set of testcolor patches ranked by human observers for watermark visibility.

A full color visibility model can be a powerful tool to measurevisibility of an image watermark. Watermarks used for packaging can beinserted in the chrominance domain to obtain the best robustness perunit visibility. A chrominance image watermark is preferably embedded ina way that the color component in the cover image is minimally alteredand is hardly noticeable, due to human vision system's low sensitivityto color changes.

One example of a color visibility model is discussed relative to SpatialCIELAB (S-CIELAB). The accuracy of this model was tested by comparing itto human subjective tests on a set of watermarked color patches. Themodel was found to significantly overestimate the visibility of somedark color patches. A correction can be applied to the model for thevariation of the human contrast sensitivity function (CSF) withluminance. After luminance correction, better correlation was obtainedwith the subjective tests.

The luminance and chrominance CSF of the human visual system has beenmeasured for various retinal illumination levels. The luminance CSFvariation was measured by Van Nes (1967) and the chrominance CSFvariation by van der Horst (1969). These measurements show a variationin peak sensitivity of about a factor of 8 for luminance and 5 forchrominance over retinal illumination levels which change by about afactor of 100.

Since the retinal illumination can change by about a factor of 100between the lightest to darkest area on a page, the CSF peak sensitivityand shape can change significantly. The function is estimated by theaverage local luminance on the page, and a spatially dependent CSF isapplied to the image. This correction is similar to the luminancemasking in adaptive image dependent compression.

The luminance dependent CSF performed better than a fixed CSF in thevisibility model, when compared to subjective measurements of a set oftest color patches ranked by human observers for watermark visibility.In some cases, we use a method of applying a spatially dependent CSFwhich depends on local image luminance.

The visibility model can be used to embed watermark into images withequal visibility. During the embedding stage, the visibility model canpredict the visibility of the watermark signal and then adjust theembedding strength. The result will be an embedded image with a uniformwatermark signal visibility, with the embedding strength varyingdepending on the cover image's content.

The following documents are hereby incorporated herein by reference:Lyons, et al. “Geometric chrominance watermark embed for spot color,”Proc. Of SPIE, vol. 8664, Imaging and Printing in a Web 2.0 World IV,2013; Zhang et al. “A spatial extension of CIELAB for digitalcolor-image reproduction” Journal of the Society for Information Display5.1 (1997): 61-63; Van Nes et al. “Spatial modulation transfer in thehuman eye,” Journal of Optical Society of America, vol. 57, issue 3, pp.401-406, 1967; Van der Horst et al. “Spatiotemporal chromaticitydiscrimination,” Journal of Optical Society of America, vol. 59, issue11, 1969; and Watson, “DCTune,” Society for information display digestof technical papers XXIV, pp. 946-949, 1993.

In some cases, even better results can be achieved by combining anattention model with our above visibility model when embeddingwatermarks in color image data. An attention model generally predictswhere the human eye is drawn to when viewing an image. For example, theeye may seek out flesh tone colors and sharp contrast areas. One exampleattention model is described in Itti et al., “A Model of Saliency-BasedVisual Attention for Rapid Scene Analysis,” IEEE TRANSACTIONS ON PATTERNANALYSIS AND MACHINE INTELLIGENCE, VOL. 20, NO. 11, NOVEMBER 1998, pgs.1254-1259, which is hereby incorporated herein by reference.

High visual traffic areas identified by the attention model, which wouldotherwise be embedded with a relatively strong or equal watermarksignal, can be avoided or minimized by a digital watermark embedder.

Additional related disclosure is found in Appendix D, included as partof this specification, and which is hereby incorporated herein byreference in its entirety.

CONCLUDING REMARKS

Having described and illustrated the principles of the technology withreference to specific implementations, it will be recognized that thetechnology can be implemented in many other, different, forms. Toprovide a comprehensive disclosure without unduly lengthening thespecification, applicant hereby incorporates by reference each of theabove referenced patent documents in its entirety.

The methods, processes, components, apparatus and systems describedabove may be implemented in hardware, software or a combination ofhardware and software. For example, the watermark encoding processes andembedders may be implemented in software, firmware, hardware,combinations of software, firmware and hardware, a programmablecomputer, electronic processing circuitry, with a processor, parallelprocessors or other multi-processor configurations, and/or by executingsoftware or instructions with one or more processors or dedicatedcircuitry. Similarly, watermark data decoding or decoders may beimplemented in software, firmware, hardware, combinations of software,firmware and hardware, a programmable computer, electronic processingcircuitry, and/or by executing software or instructions with aprocessor, parallel processors or other multi-processor configurations.

The methods and processes described above (e.g., watermark embedders anddetectors) also may be implemented in software programs (e.g., writtenin C, C++, Visual Basic, Java, Python, Tcl, Perl, Scheme, Ruby,executable binary files, etc.) stored in memory (e.g., a computerreadable medium, such as an electronic, optical or magnetic storagedevice) and executed by a processor (or electronic processing circuitry,hardware, digital circuit, etc.).

While one embodiment discusses inverting the polarity in a second colorchannel (e.g., a ‘b’ channel), one could also invert the polarity in thefirst color channel (e.g., an ‘a’ channel) instead. In such a case, thefirst color channel is then preferably subtracted from the second colorchannel.

The particular combinations of elements and features in theabove-detailed embodiments (including Appendices A, B, C & D) areexemplary only; the interchanging and substitution of these teachingswith other teachings in this and the incorporated-by-reference patentdocuments are also contemplated.

What is claimed is:
 1. A digital watermarking method comprising:receiving color image data; using one or more programmed processors,applying a color visibility model to the color image data to identifyimage areas correlating to color patches likely to be degraded with anapplication of digital watermarking; and using one or more programmedprocessors, applying a pyramid based digital watermarking embedder toobtain equal visibility, in which the digital watermarking embedder isconfigured, at least in part, based on results obtained from applyingthe color visibility model.
 2. A digital watermarking method comprising;obtaining color image data; applying a visibility model to the colorimage data to identify image areas correlating to color patches likelyto be degraded with an application of digital watermarking; using one ormore programmed processors applying an attention model to the colorimage data, said applying resulting in an identification of image areaslikely to drawn attention of human visual attention; using one or moreprogrammed processors, applying a digital watermark embedder to thecolor image data, in which the digital watermark embedder is configured,at least in part, based on results obtained from applying both thevisibility model and the attention model.
 3. An apparatus comprising: aninput to obtain color image data; memory storing a visibility model andan attention model; and one or more processors configured for: applyingthe visibility model to the color image data to identify image areascorrelating to color patches likely to be degraded with an applicationof digital watermarking; applying the attention model to the color imagedata, the applying resulting in predicting image areas likely to drawattention of human visual traffic areas; modifying the color image datawith digital watermarking, in which the digital watermarking is based onresults obtained from both applying the visibility model and applyingthe attention model.